@benoit74 Just to be clear, glad to see you working on the issue, but I don't think put webp content in path ending with .png (just an example) is a good idea at all. It is simply semantically wrong and we should not do that IMHO. Current approach works (modulo bugs - like always) and if we really want to do better we should keep track about the content mime-type (instead of relying on the extension).

Just to be clear, glad to see you working on the issue, but I don't think put webp content in path ending with .png (just an example) is a good idea at all. It is simply semantically wrong and we should not do that IMHO.

I agree, but this would mean a significant redesign of the scraper: with current architecture, as stated in the issue, we cannot know at HTML rewriting time what the result of image download/conversion will be ; for this we need to download the image and try the reencoding, which is currently done at a totally different stage.

For now I prefer to have a scraper producing working ZIMs under all conditions with some semantic incoherence invisible to 99% of our users, rather than having non-working ZIMs like #2088. I do not mind to open an issue to fix this semantic incoherence on the medium / long term. For the record, this semantic incoherence is already present since "forever" in S3 keys used to cache image and we lived pretty well with it.

Sample ZIMs:

• wikipedia_hi_basketball_maxi_2025-01.zim : second run on dev S3 bucket (i.e. all images are coming from the S3 cache)
• psychonautwiki_en_all_maxi_2025-01.zim : first run on dev S3 bucket (i.e. all images are coming from online)

For now I prefer to have a scraper producing working ZIMs under all conditions with some semantic incoherence invisible to 99% of our users, rather than having non-working ZIMs like #2088. I do not mind to open an issue to fix this semantic incoherence on the medium / long term. For the record, this semantic incoherence is already present since "forever" in S3 keys used to cache image and we lived pretty well with it.

It's not and should be any incoherence in S3 because the entry is tagged "webp" AFAIK.

It's not and should be any incoherence in S3 because the entry is tagged "webp" AFAIK.

S3 key is computed from online URL directly without any logic handling webp conversion:

Codecov Report

Attention: Patch coverage is 75.55556% with 33 lines in your changes missing coverage. Please review.

Project coverage is 75.19%. Comparing base (75cf0fe) to head (88bbeec).

❌ Your patch check has failed because the patch coverage (75.55%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

@@            Coverage Diff             @@
##             main    #2143      +/-   ##
==========================================
- Coverage   75.98%   75.19%   -0.79%     
==========================================
  Files          41       41              
  Lines        3202     3213      +11     
  Branches      706      704       -2     
==========================================
- Hits         2433     2416      -17     
- Misses        655      676      +21     
- Partials      114      121       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.